Maximum-entropy from the probability calculus: exchangeability, sufficiency

The classical maximum-entropy principlemethod (Jaynes 1963) appears in the probability calculus as an approximation of a particular model by exchangeability or a particular model by sufficiency. The approximation from the exchangeability model can be inferred from an analysis by Jaynes (1996) and to some extent from works on entropic priors (Rodríguez 1989; 2002; Skilling 1989a; 1990). I tried to show it explicitly in a simple context (Porta Mana 2009). The approximation from the sufficiency model can be inferred from Bernardo & Smith (2000 § 4.5) and Diaconis & Freedman (1981) in combination with the KoopmanPitman-Darmois theorem (see references in § 3). In this note I illustrate how either approximations arises, in turn, and then give a heuristic synopsis of both. At the end I discuss some questions: Prediction or retrodiction? Which of the two models is preferable? (the exchangeable one.) How good is the maximum-entropy approximation? Is this a “derivation” of maximum-entropy? I assume that you are familiar with: the maximum-(relative-)entropy method (Jaynes 1957a; much clearer in Jaynes 1963; Sivia 2006; Hobson et al. 1973), especially the mathematical form of its distributions and its prescription “expectations = empirical averages”; the probability calculus (Jaynes 2003; Hailperin 1996; Jeffreys 2003; Lindley 2014); the basics of models by exchangeability and sufficiency (Bernardo et al. 2000 ch. 4), although I’ll try to explain the basic ideas behind them – likely you’ve often worked with them even if you’ve never heard of them under these names.